Keyword [GSNN] [GGNN]
Marino K, Salakhutdinov R, Gupta A. The more you know: Using knowledge graphs for image classification[J]. arXiv preprint arXiv:1612.04844, 2016.
1. Overview
1.1. Motivation
- human has ability to acquire knowledge about the world and use knowledge to reason about the visual world
- Gated Graph Neural Network (GGNN) suffers computational issues when graph is large
In this paper
- proposed Graph Search Neural Network (GSNN) mitigates the computational issues
- using knowledge improves performance on image classification
1.2. Contribution
- GSNN. incorporate potentially large knowledge graph into end-to-end system
2. Methods
2.1. GGNN
- input. graph of N nodes
output. every graph node or global output
h_{v}^{t}. hidden state for node v at time step t
- x_v. problem specific annotation
- A_v. adjacency matrix of graph for node v
- W, U. learned parameters
- (1). Initialization
- (2). propagation updates from adjacent nodes
(3-6). combine the information from adjacent nodes and current hidden state of the nodes to compute the next hidden state
output network (g). FC network
2.2. GSNN
- propagation net
- importance net
- output net
- start with initial nodes in the graph based on the likelihood of the concept being present (faster R-CNN)
add adjacent node in, learn a per-node score function to estimates the importance of these nodes
importance network. assign target importance value to each node, correspond to gt is 1, neighbours are γ, two-hop away γ^2
- add top P nodes to expanded set
- at final step T, compute the per-node-output and re-order and zero-pad the outputs into the final classification net
